Virtual vehicle skin

ABSTRACT

A system and method of enabling an augmented reality/virtual reality (AR/VR) device to augment image or video data using a virtual vehicle skin, wherein the method is carried out by vehicle electronics included within a vehicle, the method including: establishing a connection to the AR/VR device using a wireless communications device included in the vehicle electronics of the vehicle; and sending a virtual vehicle skin response to the AR/VR device via the established connection, wherein the AR/VR device is configured to obtain background video and to display the virtual vehicle skin over the obtained background video so that the virtual vehicle skin overlays a portion of the vehicle, and wherein the virtual vehicle skin response includes the virtual vehicle skin or virtual vehicle skin access information that can be used by the AR/VR device to derive or obtain the virtual vehicle skin.

INTRODUCTION

The present invention relates to using augmented reality/virtual reality(AR/VR) devices to present a virtual vehicle skin over images depictinga vehicle.

Vehicles include hardware and software capable of various multimediafunctions, including playing audio received from radio stations, mobiledevices (e.g., smartphones, tablets), many of which utilize wirelesscommunications, such as short-range wireless communications (SRWC).Additionally, vehicles include numerous vehicle system modules (VSMs)that can be used to provide information to other electronic computingdevices, such as personal mobile devices.

SUMMARY

According to one aspect of the invention, there is provided a method ofenabling an augmented reality/virtual reality (AR/VR) device to augmentimage or video data using a virtual vehicle skin, wherein the method iscarried out by vehicle electronics included within a vehicle, the methodincluding: establishing a connection to the AR/VR device using awireless communications device included in the vehicle electronics ofthe vehicle; and sending a virtual vehicle skin response to the AR/VRdevice via the established connection, wherein the AR/VR device isconfigured to obtain image or video data and to display the virtualvehicle skin over the obtained image or video data so that the virtualvehicle skin overlays a portion of the vehicle, and wherein the virtualvehicle skin response includes the virtual vehicle skin or virtualvehicle skin access information that can be used by the AR/VR device toobtain the virtual vehicle skin.

According to various embodiments, this method may further include anyone of the following features or any technically-feasible combination ofthese features:

-   -   the virtual vehicle skin is configured to be inserted into        captured image or video data such that the virtual vehicle skin        covers a portion of an interior cabin of the vehicle;    -   receiving a first virtual vehicle skin request from the AR/VR        device, sending a second virtual vehicle skin request to one or        more remote servers in response to receiving the first virtual        vehicle skin request, and receiving the virtual vehicle skin or        the virtual vehicle skin access information;    -   the virtual vehicle skin response includes the virtual vehicle        skin;    -   the digital representation of the virtual vehicle skin is        obtained from memory included in the vehicle electronics;    -   the virtual vehicle skin response includes virtual vehicle skin        access information that includes information usable by the AR/VR        device for obtaining the virtual vehicle skin from a remote        server;    -   the virtual vehicle skin access information includes a virtual        vehicle skin token that, when presented to the remote server,        results in the remote server sending the virtual vehicle skin to        the AR/VR device;    -   the AR/VR device is a pair of wearable electronic augmented        reality/virtual reality (AR/VR) goggles configured for visually        presenting augmented reality or virtual reality media content to        a user; and/or    -   the AR/VR device is a personal AR/VR device.

According to another aspect of the invention, there is provided a methodof configuring an augmented reality/virtual reality (AR/VR) device toaugment image or video data using a virtual vehicle skin, the methodincluding: capturing image or video data using a camera included on theAR/VR device; receiving a virtual vehicle skin advertisement messagefrom a vehicle, wherein the vehicle uses a wireless communicationsdevice to send the virtual vehicle skin advertisement message; inresponse to receiving the virtual vehicle skin advertisement message,sending a virtual vehicle skin request, wherein the virtual vehicle skinrequest includes information from the virtual vehicle skin advertisementmessage; and receiving a virtual vehicle skin response, wherein thevirtual vehicle skin response includes a digital representation of thevirtual vehicle skin or information that can be used by the AR/VR deviceto derive the digital representation of the virtual vehicle skin.

According to various embodiments, this method may further include anyone of the following features or any technically-feasible combination ofthese features:

-   -   the virtual vehicle skin request is sent to the vehicle, and        wherein the virtual vehicle skin response is sent by the        vehicle;    -   the virtual vehicle skin response includes virtual vehicle skin        access information, and wherein the method further includes the        steps of: sending a second virtual vehicle skin request to a        remote server using the virtual vehicle skin access information;        and receiving a second virtual vehicle skin response that        includes the digital representation of the virtual vehicle skin;    -   the virtual vehicle skin includes a plurality of virtual vehicle        skin components, and wherein the method further includes the        steps of: determining to display at least one virtual vehicle        skin component of the plurality of virtual vehicle skin        components based on the image or video data; and rendering at        least part of the at least one virtual vehicle skin component on        a display of the AR/VR device;    -   before rendering the at least part of the at least one virtual        vehicle skin component on the display of the AR/VR device,        modifying the at least one virtual vehicle skin component based        on the one or more captured image or video data;    -   the virtual vehicle skin includes virtual exterior environment        graphics, and wherein the method further includes the steps of:        determining exterior areas that are exterior to the vehicle and        within the captured image or video data; and rendering the        virtual exterior environment graphics within the exterior areas;    -   determining an orientation of the AR/VR device using an        accelerometer included on the AR/VR device;    -   the vehicle is configured to determine a position of the AR/VR        device within the vehicle and, thereafter, to obtain the virtual        vehicle skin based on the position of the AR/VR device.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the invention will hereinafter be describedin conjunction with the appended drawings, wherein like designationsdenote like elements, and wherein:

FIG. 1 is a block diagram depicting an embodiment of a communicationssystem that is capable of utilizing the method disclosed herein;

FIG. 2 is a perspective view depicting an embodiment of an interior of avehicle;

FIG. 3 is a perspective view depicting an embodiment of an interior of avehicle with a virtual vehicle skin applied to the interior of thevehicle;

FIG. 4 is a flowchart of an embodiment of a method of broadcasting anindication that virtual vehicle skins are available for use withaugmented reality/virtual reality (AR/VR) devices;

FIG. 5 is a flowchart of an embodiment of a method of sending virtualvehicle skin information to an AR/VR device; and

FIG. 6 is a flowchart of an embodiment of a method of presenting virtualvehicle skins over a corresponding area of a vehicle using an AR/VRdevice.

DETAILED DESCRIPTION

The system and method described below enables an augmentedreality/virtual reality (AR/VR) device to render virtual vehicle skinson a video or image such that the virtual vehicle skin is presented overan area corresponding to a vehicle. AR/VR devices can use camera(s) tocapture an image or video of an area surrounding a user and, then, torender graphics over certain parts of the captured image or video. Inthis way, the AR/VR device can display graphics over real-life images orvideo of a user's surroundings such that the graphics supplement theuser's visual perception of the surrounding environment. Thesesupplemental graphics can be used to provide aesthetically pleasinggraphics, as well as to provide information to the user. For example,virtual vehicle skins can be developed and displayed by AR/VR devices inconjunction with video or images of the user's environment to provide aperception that the vehicle looks as depicted in the virtual vehicleskins. AR/VR devices can use a camera that, when worn by a user, facesan area in front of the user (the “AR/VR device field of view”). In oneembodiment, the AR/VR device is a pair of AR or VR goggles that areconfigured to fit around a user's head and that includes a display that,when the goggles are properly worn, is positioned in front of the user'seyes or field of vision. The AR/VR device can download a virtual vehicleskin that is associated with the vehicle and, when the vehicle is withinthe AR/VR device field of view, the AR/VR device can display the virtualvehicle skin on captured image or video in an area corresponding to thevehicle or a particular component of the vehicle. In many embodiments,the method and system discussed below can be used to augment an areawithin the interior of the vehicle, such as within an interior vehiclecabin.

In some embodiments, virtual vehicle skins can be developed based on alayout or design of a particular vehicle and, then, the virtual vehicleskins can be sent to AR/VR devices at or near the vehicle so that theAR/VR devices can display the virtual vehicle skins over the vehicle.Since the virtual vehicle skins correspond to a layout of the vehicle,the virtual vehicle skins can be presented over a corresponding area ofthe vehicle while still conveying the general layout of the vehicle sothat the user can still operate the vehicle even when the virtualvehicle graphics are overlaid. For example, a virtual vehicle skin canbe developed for an interior of a vehicle that includes various vehiclecontrols. The skin can be developed to provide various aesthetic effectswhile doing so within the confines of the vehicle layout, which caninclude designing the virtual vehicle skin to include the same physicalshape and layout of the vehicle. Thus, when the virtual vehicle skin ispresented over areas of the vehicle, the functional aspects of thevehicle, such as the various vehicle controls, can still be seen andused by the user and, additionally, the general layout may still beretained so that the user may still be aware of the physical confines ofthe vehicle.

In one embodiment, a user can access a virtual vehicle skin catalog andselect various virtual vehicle skins that the user desires to haveassociated with their vehicle. The user can select and/or purchase suchvirtual vehicle skins and, thereafter, the virtual vehicle skins can beassociated with a particular vehicle of the user. The virtual vehicleskins can then be sent to the vehicle, which can store the skins and/orsend the skins to AR/VR devices. In one particular embodiment, thevehicle can broadcast a virtual vehicle skin availability message thatindicates to the AR/VR devices that virtual vehicle skins are availablefor the vehicle. The vehicle can then receive a request for a particularskin and, in response thereto, the vehicle can obtain the virtualvehicle skin and send the virtual vehicle skin to the AR/VR device. Insome embodiments, obtaining the virtual vehicle skin can include sendinga request for the skin to a remote facility and then receiving thevirtual vehicle skin from the remote facility. In other embodiments,obtaining the virtual vehicle skin can include recalling the virtualvehicle skin from a memory device located in the vehicle.

As mentioned above, the vehicle can periodically transmit a virtualvehicle skin availability message. Upon receiving this indication at anAR/VR device, the AR/VR device can determine whether to obtain aparticular virtual vehicle skin and, if so, the AR/VR device can send avirtual vehicle skin request to the vehicle or to a remote facility. Inone embodiment, the virtual vehicle skin request can be generated basedon information (e.g., VIN, virtual vehicle skin ID) in the virtualvehicle skin availability message and then the request can be sent to aremote facility. The remote facility can respond by sending therequested virtual vehicle skin to the AR/VR device. In otherembodiments, the virtual vehicle skin request can be sent to thevehicle, which can then obtain the virtual vehicle skin(s) and send themto the AR/VR device. Once the AR/VR device receives the virtual vehicleskin, the AR/VR device can present the virtual vehicle skin on capturedimage or video of the surrounding area so that graphics of the virtualvehicle skin are shown over the vehicle or a vehicle component.

With reference to FIG. 1, there is shown an operating environment thatcomprises a communications system 10 and that can be used to implementthe method disclosed herein. Communications system 10 generally includesa vehicle 12 with a wireless communications device 30, augmentedreality/virtual reality (AR/VR) devices 14 and 16, a constellation ofglobal navigation satellite system (GNSS) satellites 60, one or morewireless carrier systems 70, a land communications network 76, acomputer or server 78, and a remote facility 80. It should be understoodthat the disclosed method can be used with any number of differentsystems and is not specifically limited to the operating environmentshown here. Also, the architecture, construction, setup, and generaloperation of the system 10 and its individual components are generallyknown in the art. Thus, the following paragraphs simply provide a briefoverview of one such communications system 10; however, other systemsnot shown here could employ the disclosed method as well.

Wireless carrier system 70 may be any suitable cellular telephonesystem. Carrier system 70 is shown as including a cellular tower 72;however, the carrier system 70 may include one or more of the followingcomponents (e.g., depending on the cellular technology): cellulartowers, base transceiver stations, mobile switching centers, basestation controllers, evolved nodes (e.g., eNodeBs), mobility managemententities (MMEs), serving and PGN gateways, etc., as well as any othernetworking components required to connect wireless carrier system 70with the land network 76 or to connect the wireless carrier system withuser equipment (UEs, e.g., which can include telematics equipment invehicle 12). Carrier system 70 can implement any suitable communicationstechnology, including GSM/GPRS technology, CDMA or CDMA2000 technology,LTE technology, etc. In general, wireless carrier systems 70, theircomponents, the arrangement of their components, the interaction betweenthe components, etc. is generally known in the art.

Apart from using wireless carrier system 70, a different wirelesscarrier system in the form of satellite communication can be used toprovide uni-directional or bi-directional communication with thevehicle. This can be done using one or more communication satellites(not shown) and an uplink transmitting station (not shown).Uni-directional communication can be, for example, satellite radioservices, wherein programming content (news, music, etc.) is received bythe uplink transmitting station, packaged for upload, and then sent tothe satellite, which broadcasts the programming to subscribers.Bi-directional communication can be, for example, satellite telephonyservices using the one or more communication satellites to relaytelephone communications between the vehicle 12 and the uplinktransmitting station. If used, this satellite telephony can be utilizedeither in addition to or in lieu of wireless carrier system 70.

Land network 76 may be a conventional land-based telecommunicationsnetwork that is connected to one or more landline telephones andconnects wireless carrier system 70 to remote facility 80. For example,land network 76 may include a public switched telephone network (PSTN)such as that used to provide hardwired telephony, packet-switched datacommunications, and the Internet infrastructure. One or more segments ofland network 76 could be implemented through the use of a standard wirednetwork, a fiber or other optical network, a cable network, power lines,other wireless networks such as wireless local area networks (WLANs), ornetworks providing broadband wireless access (BWA), or any combinationthereof.

Computers 78 (only one shown) can be some of a number of computersaccessible via a private or public network such as the Internet. Eachsuch computer 78 can be used for one or more purposes, such as forpurchasing or selecting various virtual vehicle skins for use withvehicle 12. Other such accessible computers 78 can be, for example: aservice center computer where diagnostic information and other vehicledata can be uploaded from the vehicle; a client computer used by thevehicle owner or other subscriber for such purposes as accessing orreceiving vehicle data or to setting up or configuring subscriberpreferences or controlling vehicle functions; a car sharing server whichcoordinates registrations from a plurality of users who request to use avehicle as part of a car sharing service; or a third party repository toor from which vehicle data or other information is provided, whether bycommunicating with the vehicle 12, remote facility 80, or both. Acomputer 78 can also be used for providing Internet connectivity such asDNS services or as a network address server that uses DHCP or othersuitable protocol to assign an IP address to vehicle 12. In oneembodiment, computers 78 can be used to carry out the method discussedherein; in other embodiments, the method can be carried out by serversor other computing devices at remote facility 80, as discussed morebelow; and, it yet another embodiment, the method can be carried out bya combination of computers 78 and servers at remote facility 80.

Remote facility 80 may be designed to provide the vehicle electronics 20and/or AR/VR devices 14,16 with a number of different system back-endfunctions through use of one or more electronic servers and, in manycases, may be a vehicle backend services facility that providesvehicle-related backend functionality. The remote facility 80 includesservers (vehicle backend services servers) 82 and databases 84, whichmay be stored on a plurality of memory devices. Also, remote facility 80can include one or more switches, live advisors, an automated voiceresponse system (VRS), all of which are known in the art. Remotefacility 80 may include any or all of these various components and,preferably, each of the various components are coupled to one anothervia a wired or wireless local area network. Remote facility 80 mayreceive and transmit data via a modem connected to land network 76. Datatransmissions may also be conducted by wireless systems, such as IEEE802.11x, GPRS, and the like. Those skilled in the art will appreciatethat, although only one remote facility 80 and one computer 78 aredepicted in the illustrated embodiment, numerous remote facilities 80and/or computers 78 may be used.

Servers 82 can be computers or other computing devices that include atleast one processor and that include memory. The processors can be anytype of device capable of processing electronic instructions includingmicroprocessors, microcontrollers, host processors, controllers, vehiclecommunication processors, and application specific integrated circuits(ASICs). The processors can be dedicated processors used only forservers 82 or can be shared with other systems. The at least oneprocessor can execute various types of digitally-stored instructions,such as software or firmware programs stored in the memory (e.g.,EEPROM, RAM, ROM), which enable the servers 82 to provide a wide varietyof services. For instance, the at least one processor can executeprograms or process data to carry out at least a part of the methoddiscussed herein. For network communications (e.g., intra-networkcommunications, inter-network communications including Internetconnections), the servers can include one or more network interfacecards (NICs) (including wireless NICs (WNICs)) that can be used totransport data to and from the computers. These NICs can allow the oneor more servers 82 to connect with one another, databases 84, or othernetworking devices, including routers, modems, and/or switches. In oneparticular embodiment, the NICs (including WNICs) of servers 82 mayallow SRWC connections to be established and/or may include Ethernet(IEEE 802.3) ports to which Ethernet cables may be connected to that canprovide for a data connection between two or more devices. Remotefacility 80 can include a number of routers, modems, switches, or othernetwork devices that can be used to provide networking capabilities,such as connecting with land network 76 and/or cellular carrier system70. In one embodiment, servers 82 include an application that carriesout at least some of the method illustrated below in FIG. 4.

Databases 84 can be stored on a plurality of memory, such as RAM, othertemporary powered memory, any non-transitory computer-readable medium(e.g., EEPROM), or any other electronic computer medium that stores someor all of the software needed to carry out the various external devicefunctions discussed herein. Also, one or more databases at the remotefacility can store account information, as well as virtual vehicle skinassociation information and/or the virtual vehicle skins. As usedherein, “virtual vehicle skin association information” refers to thatinformation that associates a particular vehicle with a particularvirtual vehicle skin. Digital representations of the virtual vehicleskins can be stored at databases 84. Also, a vehicle informationdatabase can be included that stores information pertaining to one ormore vehicles, such as vehicle identification numbers (VINs), virtualvehicle skin identifiers, vehicle layout types or features, as well asvarious other data pertaining to the vehicle. The vehicle layout typesor features can indicate general layout of the vehicle and maycorrespond to a model-year (e.g., 2018 Cruze®). Moreover, particularfeatures of a particular vehicle can be stored in databases 84, such aswhether the particular vehicle includes one or more optional features.

Smartphone 14 is an augmented reality/virtual reality (AR/VR) devicethat is capable of providing media content to one or more interfaces(either included in smartphone 14 or included at another deviceconnected to smartphone 14). As used herein, “AR/VR device” refers tothose devices that are configured to obtain and present AR/VR mediacontent and that include suitable hardware and processing capabilitiesfor presenting AR/VR media content. Non-limiting examples of mediacontrol devices are a smartphone, personal computer (PC), laptop,tablet, smart television, and AR/VR goggles or displays. In someembodiments, the smartphone (media control device) 14 may include:hardware, software, and/or firmware enabling cellular telecommunicationsand short-range wireless communications (SRWC) as well as other mobiledevice applications. As used herein, a personal AR/VR device is an AR/VRdevice that is portable by a user and where the portability of thedevice is at least partly dependent on the user, such as a wearabledevice (e.g., a smartwatch, AR/VR goggles), an implantable device, or ahandheld device (e.g., a smartphone, a tablet, a laptop). And, as usedherein, a short-range wireless communications (SRWC) device is a devicecapable of SRWC and that includes the requisite SRWC circuitry toperform such SRWC.

In many embodiments, the hardware of the AR/VR devices, includingsmartphone 14, may comprise a processor and memory (e.g., non-transitorycomputer readable medium configured to operate with the processor) forstoring the software, firmware, etc. In one embodiment, the AR/VRdevices can include a SRWC circuit, such as the one discussed below withrespect to wireless communications device 30 of vehicle 12. The AR/VRdevice's processor and memory may enable various software applications,which may be preinstalled or installed by the user (or manufacturer)(e.g., having a software application or graphical user interface (GUI)).One implementation of an AR/VR device application may enable the AR/VRdevice (e.g., smartphone 14) to receive media content (e.g., virtualvehicle skins) via land network 76 and/or cellular carrier system 70,and to play or otherwise present the media content to a user via one ormore user-device interfaces at the AR/VR device. In one embodiment,AR/VR device 14 may include a display that can be used to visuallypresent media content to a user and may include an audio system that canbe used to audibly present media content to a user. Other playbackcomponents or devices can be included and/or controlled by the AR/VRdevice 14, such as a tactile or haptic device.

In some embodiments, a visual display head mount can be used inconjunction with AR/VR device (e.g., smartphone 14) to mount the devicein front of the user's field of vision, as well as enclosing theperipheral field of view of the user. The visual display head mount canbe a separately sold and manufactured device that includes a slot orthat is otherwise configured for holding AR/VR devices in an orientationsuch that, when the device is placed within the slot (or otherwiseattached to the visual display head mount) and worn properly by theuser, the display of the AR/VR device is within the user's field ofvision. Such a visual display head mount that is used in conjunctionwith a suitable AR/VR device can be used to transform an ordinary AR/VRdevice (e.g., smartphone) into a pair of AR/VR goggles. As those skilledin the art will appreciate, even though AR/VR headsets may be referredto as a pair of goggles, only a single display is required, rather thana pair of displays as the name would ostensibly imply.

The pair of Augmented Reality/Virtual Reality (AR/VR) goggles 16 is anAR/VR device that can be used to obtain, process, and presentaudiovisual content to a user, such as virtual vehicle skins. The pairof AR/VR goggles 16 is a head-mounted display that includes a processorand memory, as well as computer instructions (e.g., software, firmware)that enables AR/VR media content to be displayed. And, in someembodiments, the AR/VR goggles 16 can be an AR/VR headset that ismanufactured to include AR/VR goggles integrated with a head mount andthat includes an audio speaker. The AR/VR goggles 16 can also includeother components, such as a SRWC circuit (similar to SRWC circuit 32discussed below), an accelerometer (or orientation sensors), one or morecameras or other lens capable of receiving and digitizing light signals,a battery, speakers or headphones for presenting audio content, andvarious other hardware or software components. The AR/VR goggles can beconfigured for use with applications developed according to a softwaredevelopment kit (SDK) that is configured specifically for AR/VR devices,or that can support AR/VR functionality.

Additionally, as mentioned above, the AR/VR goggles can include speakersor headphones that can present audio content to the user via soundwaves.Such audio content may be a part of audiovisual media content, and theaudio and video output of the AR/VR goggles can be synced. In otherembodiments, the AR/VR goggles may only include a video output and mayconnect with other user-device interfaces, such as those included in thevehicle, to output the audio content. And, as will be discussed in moredetail below, the AR/VR goggles can send virtual vehicle skin requeststo the vehicle in an attempt to obtain one or more virtual vehicle skinsfor use in augmenting a field of view of the AR/VR goggles that includesan area of the vehicle. These virtual vehicle skin requests can bebundled into a single request that is sent by the AR/VR gogglesinitially (e.g., at the beginning of an immersive media contentexperience), or may be sent at times corresponding to when the requestedvehicle functionality is desired.

Vehicle 12 is depicted in the illustrated embodiment as a passenger car,but it should be appreciated that any other vehicle includingmotorcycles, trucks, sports utility vehicles (SUVs), recreationalvehicles (RVs), marine vessels, aircraft, etc., can also be used. Someof the vehicle electronics 20 are shown generally in FIG. 1 and includesa global navigation satellite system (GNSS) module 22, body control unit(BCM) 24, other vehicle system modules (VSMs) 26, and a wirelesscommunications device 30. Some or all of the different vehicleelectronics may be connected for communication with each other via oneor more communication busses, such as bus 28. Communications bus 28provides the vehicle electronics with network connections using one ormore network protocols. Examples of suitable network connections includea controller area network (CAN), a media oriented system transfer(MOST), a local interconnection network (LIN), a local area network(LAN), and other appropriate connections such as Ethernet or others thatconform with known ISO, SAE and IEEE standards and specifications, toname but a few.

The vehicle 12 can include numerous vehicle system modules (VSMs) aspart of vehicle electronics 20, such as the GNSS module 22, BCM 24,wireless communications device 30, and vehicle-user interfaces 52-58, aswill be described in detail below. The vehicle 12 can also include otherVSMs 26 in the form of electronic hardware components that are locatedthroughout the vehicle and, which may receive input from one or moresensors and use the sensed input to perform diagnostic, monitoring,control, reporting, and/or other functions. Each of the VSMs 26 ispreferably connected by communications bus 28 to the other VSMs, as wellas to the wireless communications device 30, and can be programmed torun vehicle system and subsystem diagnostic tests. One or more VSMs 26may periodically or occasionally have their software or firmware updatedand, in some embodiments, such vehicle updates may be over the air (OTA)updates that are received from a computer 78 or remote facility 80 vialand network 76 and communications device 30. As is appreciated by thoseskilled in the art, the above-mentioned VSMs are only examples of someof the modules that may be used in vehicle 12, as numerous others arealso possible.

Wireless communications device 30 is capable of communicating data viashort-range wireless communications (SRWC) and/or via cellular networkcommunications through use of a cellular chipset 34, as depicted in theillustrated embodiment. In the illustrated embodiment, wirelesscommunications device 30 includes an SRWC circuit 32, a cellular chipset34, a processor 36, memory 38, and antennas 40 and 50. In oneembodiment, wireless communications device 30 may be a standalone moduleor, in other embodiments, device 30 may be incorporated or included as apart of one or more other vehicle system modules, such as a center stackmodule (CSM), body control module (BCM) 24, an infotainment module, atelematics unit, a head unit, and/or a gateway module. In someembodiments, the device 30 can be implemented as an OEM-installed(embedded) or aftermarket device that is installed in the vehicle.

Wireless communications device 30 can be configured to communicatewirelessly according to one or more wireless protocols, includingshort-range wireless communications (SRWC) such as any of the IEEE802.11 protocols, Wi-Fi™ WiMAX™, ZigBee™, Wi-Fi Direct™, Bluetooth™,Bluetooth™ Low Energy (BLE), or near field communication (NFC). As usedherein, Bluetooth™ refers to any of the Bluetooth™ technologies, such asBluetooth Low Energy™ (BLE), Bluetooth™ 4.1, Bluetooth™ 4.2, Bluetooth™5.0, and other Bluetooth™ technologies that may be developed. As usedherein, Wi-Fi™ or Wi-Fi™ technology refers to any of the Wi-Fi™technologies, such as IEEE 802.11b/g/n/ac or any other IEEE 802.11technology. The short-range wireless communication (SRWC) circuit 32enables the wireless communications device 30 to transmit and receiveSRWC signals, such as BLE signals. The SRWC circuit may allow the device30 to connect to another SRWC device. Additionally, in some embodiments,the wireless communications device may contain a cellular chipset 34thereby allowing the device to communicate via one or more cellularprotocols, such as those used by cellular carrier system 70. In otherembodiments, the cellular chipset 34 may be incorporated into anotherVSM, such as a separate telematics unit.

Wireless communications device 30 may enable vehicle 12 to be incommunication with one or more remote networks (e.g., one or morenetworks at remote facility 80 or computers 78) via packet-switched datacommunication. This packet-switched data communication may be carriedout through use of a non-vehicle wireless access point that is connectedto a land network via a router or modem. When used for packet-switcheddata communication such as TCP/IP, the communications device 30 can beconfigured with a static IP address or can be set up to automaticallyreceive an assigned IP address from another device on the network suchas a router or from a network address server.

Packet-switched data communications may also be carried out via use of acellular network that may be accessible by the device 30. Communicationsdevice 30 may, via cellular chipset 34, communicate data over wirelesscarrier system 70. In such an embodiment, radio transmissions may beused to establish a communications channel, such as a voice channeland/or a data channel, with wireless carrier system 70 so that voiceand/or data transmissions can be sent and received over the channel.Data can be sent either via a data connection, such as via packet datatransmission over a data channel, or via a voice channel usingtechniques known in the art. For combined services that involve bothvoice communication and data communication, the system can utilize asingle call over a voice channel and switch as needed between voice anddata transmission over the voice channel, and this can be done usingtechniques known to those skilled in the art.

Processor 36 can be any type of device capable of processing electronicinstructions including microprocessors, microcontrollers, hostprocessors, controllers, vehicle communication processors, andapplication specific integrated circuits (ASICs). It can be a dedicatedprocessor used only for communications device 30 or can be shared withother vehicle systems. Processor 36 executes various types ofdigitally-stored instructions, such as software or firmware programsstored in memory 38, which enable the device 30 to provide a widevariety of services. For instance, processor 36 can execute programs orprocess data to carry out at least a part of the method discussedherein. Memory 38 may include RAM, other temporary powered memory, anynon-transitory computer-readable medium (e.g., EEPROM), or any otherelectronic computer medium that stores some or all of the softwareneeded to carry out the various external device functions discussedherein. Similar components to those previously described (processor 36and/or memory 38, as well as SRWC circuit 32 and cellular chipset 34)can be included in AR/VR devices 14 and/or 16.

In one embodiment, the wireless communications device 30 may operateboth when the vehicle is in a powered on state and when the vehicle isin a powered off state. As used herein, a “powered on state” is a stateof the vehicle in which the ignition or primary propulsion system of thevehicle is powered on and, as used herein, a “powered off state” is astate of the vehicle in which the ignition or primary propulsion systemof the vehicle is not powered on. The operation or state of the wirelesscommunications device 30 may be controlled by another vehicle systemmodule, such as by a body control module or by an infotainment module.In the powered on state, the wireless communications device 30 mayalways be kept “on” or supplied with power from a vehicle battery orother power source. In the powered off state, the wirelesscommunications device 30 may be kept in a low-power mode or may besupplied power periodically so that device 30 may wake up and performoperations.

The vehicle may use the wireless communications device 30 to detectother SRWC devices, such as AR/VR devices 14,16. A connection betweenthe wireless communications device 30 and one or more devices 14,16 mayallow for the operation of various vehicle-device functionality.Vehicle-device functionality (or function) refers to any function of thevehicle that may be complimented, improved, supported, or carried outthrough a AR/VR device; any function of the AR/VR device that may becomplimented, improved, supported, or carried out through the vehicle;or any other function that may be carried out using the vehicle and oneor more AR/VR devices. For example, vehicle-device functionality caninclude using the AR/VR device 14 to provide the vehicle with a contactlist that may be displayed on visual display 58, audio/visual mediacontent to be played via speakers 54 or display 58, navigationalinformation (e.g., start and/or end locations), and/or vehicle commandsor instructions that direct the vehicle to perform some operation. Otherexamples of vehicle-device functionality include using the vehicleelectronics to provide the AR/VR device with hands-free calling, such asthrough use of vehicle-user interfaces to place, receive, and carry outvoice calls; sending information to the AR/VR device, such asgeographical information to the AR/VR device 14 (such as informationobtained from the GPS module 22) or vehicle diagnostic information orcodes; and carrying out commands (e.g., commands to perform certainvehicle functions) received at the vehicle from the personal contentplayback device.

The wireless communications device 30 may be configured to listen forone or more wireless signals, such as those that may be sent by AR/VRdevice 14. In one embodiment, wireless communications device 30 mayreceive a connection request message from AR/VR device 14 or 16.Thereafter, the wireless communications device 30 can establish a SRWCconnection with the requesting device via a four-way handshake and/orother connection establishment processes, such as those that establish ashared secret or one or more digital keys or other credentials used forsecuring the SRWC connection.

In one embodiment, wireless communications device 30 may be set to adiscovery mode when the vehicle is switched to a powered on state. Asused herein, a discovery mode is an operating mode for a SRWC device inwhich the SRWC device sends out wireless messages that typically areintended to discover or detect other SRWC devices using a SRWC protocolor technology. The discovery mode may include periodically orintermittently sending out messages or advertisements in an attempt toelicit a response from other SRWC devices in range. In one embodiment,the wireless communications device 30 may transmit a Bluetooth™ LowEnergy advertising package (e.g., an advertisement), such as an ADV_IND(an undirected advertisement) message, an ADV_DIRECT_IND (a directedadvertisement) message, or an ADV_SCAN_IND (scan-able undirectedadvertisement). For example, the wireless communications device 30 maytransmit an ADV_IND message according to a predefined or predeterminedtime interval (e.g., 30 milliseconds (ms)). Or, the device 30 may varythe time interval by randomizing the interval (e.g., randomly orpseudo-randomly selecting a time between 10 ms and 40 ms for eachadvertisement). This advertisement can include information indicatingthat virtual vehicle skins are available for use with AR/VR devices forsupplementing or augmenting a view of vehicle 12. In other embodiments,a wired communication connection can be established between AR/VRdevices 14,16 and the vehicle, such as through using Universal SerialBus (USB) cables and drivers included on the respective devices.

Global navigation satellite system (GNSS) module 22 receives radiosignals from a constellation of GNSS satellites. In one embodiment, theGNSS module 22 may be a global positioning system (GPS) module, whichmay receive GPS signals from a constellation of GPS satellites 60. GNSSmodule 22 may be used to provide navigation and other position-relatedservices to the vehicle operator. Navigation information can bepresented on the display 58 (or other display within the vehicle) or canbe presented verbally such as is done when supplying turn-by-turnnavigation. The navigation services can be provided using a dedicatedin-vehicle navigation module (which can be part of GNSS module 22), orsome or all navigation services can be done via a telematics unitinstalled in the vehicle, wherein the position information is sent to aremote location for purposes of providing the vehicle with navigationmaps, map annotations (points of interest, restaurants, etc.), routecalculations, and the like. The position information can be supplied toremote facility 80 or other remote computer system, such as computer 78,for other purposes, such as fleet management and/or for use in a carsharing service. Also, new or updated map data can be downloaded to theGNSS module 22 from the remote facility 80 via a vehicle telematicsunit.

Vehicle electronics 20 also includes a number of vehicle user interfacesthat provide vehicle occupants with a means of providing and/orreceiving information, including pushbutton(s) 52, audio system 54,microphone 56, and visual display 58. As used herein, the term“vehicle-user interface” broadly includes any suitable form ofelectronic device, including both hardware and software components,which is located on the vehicle and enables a vehicle user tocommunicate with or through a component of the vehicle. Thepushbutton(s) 52 allow manual user input into the communications device30 to provide other data, response, or control input. Audio system 54provides audio output to a vehicle occupant and can be a dedicated,stand-alone system or part of the primary vehicle audio system.According to the particular embodiment shown here, audio system 54 isoperatively coupled to both vehicle bus 28 and an entertainment bus (notshown) and can provide AM, FM and satellite radio, CD, DVD and othermultimedia functionality. This functionality can be provided inconjunction with or independent of an infotainment module. Microphone 56provides audio input to the wireless communications device 30 to enablethe driver or other occupant to provide voice commands and/or carry outhands-free calling via the wireless carrier system 70. For this purpose,it can be connected to an on-board automated voice processing unitutilizing human-machine interface (HMI) technology known in the art.Visual display or touch screen 58 is preferably a graphics display andcan be used to provide a multitude of input and output functions.Display 58 can be a touch screen on the instrument panel, a heads-updisplay reflected off of the windshield, or a projector that can projectgraphics for viewing by a vehicle occupant. Various other vehicle userinterfaces can also be utilized, as the interfaces of FIG. 1 are only anexample of one particular implementation.

Body control module (BCM) 24 is shown in the exemplary embodiment ofFIG. 1 as being electrically coupled to communication bus 28. In someembodiments, the BCM 24 may be integrated with or part of a center stackmodule (CSM) and/or integrated with wireless communications device 30.Or, the BCM may be a separate device that is connected to one anothervia bus 28. BCM 24 can include a processor and/or memory, which can besimilar to processor 34 and memory 36 of wireless communications device30, as discussed below. BCM 24 may communicate with wireless device 30and/or one or more vehicle system modules, such as an engine controlunit (ECU) (not shown), audio system 54, or other VSMs 26. BCM 24 mayinclude a processor and memory such that the BCM may direct one or morevehicle operations including, for example, controlling central locking,air conditioning, power mirrors, controlling the vehicle primary mover(e.g., engine, primary propulsion system), and/or controlling variousother vehicle modules. BCM 24 may receive data from the wirelesscommunications device 30 and, subsequently, send the data to one or morevehicle modules.

Additionally, BCM 26 may provide vehicle state information correspondingto the vehicle state or of certain vehicle components or systems. Forexample, the BCM may provide the device 30 with information indicatingwhether the vehicle's ignition is turned on, the gear the vehicle ispresently in (i.e. gear state), and/or other information regarding thevehicle. The BCM 26 can obtain information from one or more othervehicle modules to obtain this information.

With reference to FIGS. 2 and 3, there is shown an interior 100 of avehicle 12 from the perspective of a passenger in the back row of seatslooking forward through windshield 102 using an AR/VR device, such assmartphone 14 or AR/VR goggles 16. FIG. 2 is a perspective view throughthe AR/VR device without a virtual vehicle skin applied and FIG. 3 is aperspective view through the AR/VR device with a virtual vehicle skinapplied. The vehicle interior 100 includes the windshield 102, steeringwheel 104, center console or instrument panel 106 that holds touchscreendisplay 58 and wireless communications device 30, a touchscreen userinterface 108 displayed on touchscreen 58, a shifter 112, seats 120 and130, and A-pillars 140 and 142. Wireless communications device 30communicate with AR/VR devices within or near vehicle interior 100, asshown by wireless signals 144 and 146.

As shown in the illustrated embodiment of FIG. 3, a virtual vehicle skincan be presented over the interior 100 of vehicle 12, such as atparticular vehicle components. For example, virtual vehicle skin 148comprises numerous virtual vehicle skin components 150-160. The virtualvehicle skin includes virtual trim skin graphics shown at 154 and 160,as well as virtual seat skin graphics 156 and 158. Moreover, virtualA-pillar skin graphics are shown at 150 and 152, which are presentedover A-pillars 140 and 142, respectively. The virtual vehicle skincomponents can be configured to correspond to a particular vehiclecomponent, such as seats 120,130, pillars 140,142, and/or other vehicletrim or components. The virtual vehicle skin components can be renderedindividually and may be individually resized, repositioned, transformed,and/or modified individually based on movement of the AR/VR device, asthose skilled in the art will appreciate. Moreover, functional featuresof the vehicle, such as cup-holders 114, display 58, and shifter 112 arenot covered by the virtual vehicle skin so as to enable continued useand awareness of these functional vehicle components. In otherembodiments, the method (as discussed below) can include determiningwhich vehicle functional components are out of reach of the AR/VR deviceuser and, in such a case, these functional components may still becovered by vehicle skin 148 since the AR/VR device user cannot reach toaccess these components anyway. And, although a single, plaid/checkeredthemed virtual vehicle skin is depicted, various designs, themes,scenes, colors, patterns, and other visual effects can be used orincorporated into the virtual vehicle skin.

With reference to FIG. 4, there is shown a method 300 of broadcasting anindication that virtual vehicle skins are available for use withaugmented reality/virtual reality (AR/VR) devices. Method 300 can becarried out by one or more remote servers, such as remote servers 82 atremote facility 80. According to many embodiments, method 300 enables auser to specify one or more virtual vehicle skins that can be used toaugment graphics over the user's vehicle when being viewed by a userusing an AR/VR device. For example, a vehicle owner or operator may usea client device (e.g., computer 78, AR/VR device 14,16) to select one ormore virtual vehicle skins that are aesthetically pleasing to them, suchas a virtual vehicle skin that is themed as their favorite superhero,such as Batman™ or Superman™. Other virtual vehicle skins can bedeveloped for augmenting images of a vehicle, such as a jungle theme, anouter space theme, or a sports theme.

Prior to method 300, a user may use a computer, which can include anAR/VR device, to access a virtual vehicle skin catalog (i.e., list ofvirtual vehicle skins). The user may browse various skins, as well asinput vehicle information concerning the type of vehicle or the identityof the vehicle. As mentioned above, virtual vehicle skins can be fittedor designed to correspond to a layout of the vehicle and, thus, thevariety of virtual vehicle skins may vary by vehicle model ormodel-year. Vehicle information concerning the type or identity of thevehicle can be inputted into the virtual vehicle skin catalog and, then,the virtual vehicle skins can be filtered based on the inputted vehicleinformation. In one embodiment, the virtual vehicle skins can bepurchased from the virtual vehicle skin catalog and, when purchased, thevirtual vehicle skins may be downloaded to the vehicle, an AR/VR device,or other device.

Method 300 begins with step 310, wherein a virtual vehicle skinassociation request is received. The virtual vehicle skin associationrequest can be a message that indicates that a user desires to have aparticular virtual vehicle skin associated with their vehicle oraccount. This request can be generated and/or sent by the client devicethat the user is operating to view the virtual vehicle skin catalog. Inone embodiment, once a user purchases a virtual vehicle skin (orotherwise selects to have a virtual vehicle skin associated with theuser's vehicle), the virtual vehicle skin association request can begenerated and sent to the remote server.

In one embodiment, the virtual vehicle skin association request caninclude an identifier of the user's vehicle, or other vehicleinformation that can be used to obtain a list of suitable virtualvehicle skins for the user's vehicle. In one embodiment, the virtualvehicle skin association request can include a vehicle identificationnumber (VIN) that can then be used to obtain particular informationregarding the vehicle, including the model-year of the vehicle, as wellas particular optional features that are included in the vehicle. Theremote server can then use the specific vehicle information to determinewhether the selected virtual vehicle skin is suitable for use with thevehicle (see step 320).

In other embodiments, an identifier of the vehicle (e.g., VIN,model-year) can be sent to the remote server and, in response, theremote server can send information concerning various virtual vehicleskins to the client device (e.g., computer 78). The informationconcerning the various virtual vehicle skins can then be presented tothe user and, thereafter, the user can select a particular virtualvehicle skin. After selecting a particular virtual vehicle skin, theclient device can generate and send a virtual vehicle skin associationrequest to the remote server. In such an embodiment, the virtual vehicleskin association request may not include any particular informationregarding the user's vehicle 12, as this information was previously sentto the remote server. In other embodiments, the virtual vehicle skin canbe associated with a user's account and, also, the user's account can beassociated with one or more vehicles. The method 300 can then continueto step 320.

In step 320, it is determined whether a particular virtual vehicle skinis suited or configured for use with the user's vehicle. In someembodiments, this step may be carried out prior to receiving the virtualvehicle skin association request. For example, the remote server mayalready have information concerning a particular vehicle before thevirtual vehicle skin association request is received and, thus, at thistime the remote server can verify which skins are suitable for use withthe vehicle 12. In other embodiments, this determination can be madeafter receiving the virtual vehicle skin association request. Once it isdetermined that the requested virtual vehicle skin is suitable orconfigured for use with the vehicle, the method 300 continues to step330.

In step 330, records can be updated to reflect an association betweenthe vehicle and the virtual vehicle skin. As mentioned above, databasesmay be kept that store information pertaining to vehicles and to virtualvehicle skins. These databases can be updated to reflect that aparticular virtual vehicle skin is associated with a particular vehicle.In one embodiment, databases 84 can include a virtual vehicle skindatabase that includes information concerning the virtual vehicle skins,as well as vehicle-skin association information, which is informationthat indicates a particular vehicle or user is authorized to have avirtual vehicle skin presented over a captured image or video of thevehicle when using an AR/VR device. In other embodiments, this step caninclude associating the requested virtual vehicle skin with a user'saccount, which is also associated with particular vehicles. The method300 continues to step 340.

In step 340, virtual vehicle skin information concerning the virtualvehicle skin that was indicated in the virtual vehicle associationrequest can be sent to the vehicle. In one embodiment, the virtualvehicle skin information can be sent via land network 76 and cellularcarrier system 70. The vehicle can receive the virtual vehicle skininformation using wireless communications device 30, which can include acellular chipset 34. In one embodiment, the virtual vehicle skininformation can be sent to the vehicle soon after the virtual vehicleskin is purchased and/or associated with the particular vehicle. Or, inother embodiments, the virtual vehicle skin information can be sent tothe vehicle 12 in response to receiving a request from the vehicle 12,such as a virtual vehicle skin request (see step 420 of method 400 (FIG.5)).

In one embodiment, the virtual vehicle skin information can include anidentifier of the virtual vehicle skin. This identifier can then be usedby the vehicle to request the virtual vehicle skin from a remotedatabase, such as database 84. In other embodiments, the virtual vehicleskin information can include data representing the virtual vehicle skin,which can include various graphics and other information, such aspositional information or other information that can aid an AR/VR devicein properly presenting the virtual vehicle skin over the vehicle.

After the virtual vehicle skin information is received, the virtualvehicle skin information can be stored in a memory device of thevehicle. In one embodiment, the virtual vehicle skin information can bestored in memory 38 of wireless communications device 30, or the vehiclecan use other memory devices to store this information. Additionally,the vehicle can use the virtual vehicle skin information to transmit orbroadcast virtual vehicle skin availability messages intended forreception by one or more AR/VR devices. The virtual vehicle skinavailability messages can indicate that one or more particular virtualvehicle skins are available for presentation in an augmented realityview of the vehicle. This broadcasting step that is carried out by thevehicle is discussed more below, specifically in step 410 of method 400(FIG. 5).

With reference to FIG. 5, there is shown an embodiment of a method 400of sending virtual vehicle skin information to an AR/VR device. In someembodiments, method 400 can be carried out by vehicle 12 using wirelesscommunications device 30. Method 400 begins with step 410, wherein avirtual vehicle skin availability message is transmitted or broadcasted.The virtual vehicle skin availability message can be a wireless messagethat indicates virtual vehicle skins are available for download and/orpresentation over captured image or video of the vehicle using an AR/VRdevice. This indication may include specifying a service identifier(service ID) in the virtual vehicle skin availability message thatindicates a virtual vehicle skin service is available for use by theAR/VR device. And, in other embodiments, the virtual vehicle skinavailability message can include information concerning particularvirtual vehicle skins that are supported for use by an AR/VR device atthe vehicle. For example, an AR/VR device may receive a virtual vehicleskin availability message from the vehicle and, in response thereto, mayprompt a user for feedback as to whether the user would like to displaya virtual vehicle skin over the vehicle. The user can then use one ormore controls at their AR/VR device to respond, as is explained in moredetail below in step 520 (FIG. 6).

In one embodiment, the virtual vehicle skin availability message can beperiodically transmitted by the vehicle using those advertisementmessage discussed above. Or, the message may be transmitted in responseto a user configuring the vehicle to broadcast that virtual vehicleskins are available for download and/or presentation over captured imageor video of the vehicle using an AR/VR device. The method 400 continuesto step 420.

In step 420, a virtual vehicle skin request is received at the vehicle.The virtual vehicle skin request can be sent from an AR/VR device, suchas smartphone 14 or AR/VR goggles 16. The virtual vehicle skin requestcan be sent via SRWC and/or via a wired connection, such as a UniversalSerial Bus (USB) connection. In one embodiment, an SRWC connection canbe established after the virtual vehicle skin availability message isreceived by the AR/VR device and, after establishing the SRWCconnection, the AR/VR device can send the virtual vehicle skin requestto the vehicle via the established SRWC connection.

In the case of establishing a connection using a wired connection, auser can plug a cord or cable, such as a USB cable, into a correspondingport included within the vehicle and within the AR/VR device. In otherembodiments, where a SRWC connection is established, the vehicle cansend out a beacon or advertisement using SRWC circuit 32. Theadvertisement can be the same message as the virtual vehicle skinavailability message discussed in step 410, or may be a separatemessage. The connection advertisement message can include dataindicating that the vehicle desires to or can connect with an AR/VRdevice, or that the vehicle can offer virtual vehicle skin services(i.e., virtual vehicle skin(s) are associated with the vehicle, areavailable for download, and/or can be presented over captured image orvideo of the vehicle using an AR/VR device). This message can then bereceived by the AR/VR device, which can then generate a responsethereto, such as a message that initiates a four-way handshake or otherauthentication and/or shared key/secret establishment process. In otherembodiments, the AR/VR device can transmit an advertisement or beaconmessage indicating to those SRWC devices within range that the AR/VRdevice is searching for devices that can provide virtual vehicle skinservices. Once the connection is established, the AR/VR device may sendthe virtual vehicle skin request via the established connection.

In one embodiment, the virtual vehicle skin request can specify aparticular virtual vehicle skin (the “requested virtual vehicle skin”)that the AR/VR device desires to obtain for presentation to the user. Insuch an embodiment, the requested virtual vehicle skin can be specifiedby including a virtual vehicle skin identifier, which can then be usedby vehicle 12 or remote server 82 to obtain the corresponding virtualvehicle skin.

In other embodiments, the virtual vehicle skin request may be a generalvirtual vehicle skin request and may not specify a particular virtualvehicle skin, but may indicate that the AR/VR device is interested ormay be interested in receiving any virtual vehicle skin. In oneembodiment, after receiving the general virtual vehicle skin request,the SRWC connection (as discussed above) can be established. Thereafter,the vehicle can then send more specific information to the AR/VR device,such as the particular virtual vehicle skins that the vehicle has tooffer, which can include one or more virtual vehicle skin identifiersand/or representative graphics (i.e., a preview of the virtual vehicleskin). A user may then use their AR/VR device to select a particularvirtual vehicle skin. And, in some embodiments, the virtual vehicle skinrequest can also include AR/VR device location and position information,which can be information that can be used to establish a position of theAR/VR device 14,16 within vehicle interior 100 and/or an orientation ofthe AR/VR device. In one embodiment, AR/VR device 14,16 can use anaccelerometer to determine an orientation of the AR/VR device. Themethod 400 continues to step 430.

In step 430, the vehicle obtains the virtual vehicle skin that wasselected by the user. In the case where only a single virtual vehicleskin is available for use with vehicle 12, it can be said that, when theuser sends a general virtual vehicle skin request, the user has selectedthe single virtual vehicle skin. In one embodiment, the vehicle mayalready include virtual vehicle skin data stored in memory thatrepresents the virtual vehicle skin and, thus, the vehicle may simplyrecall this data from memory, such as memory 38. In other embodiments,the vehicle may include virtual vehicle skin information, such as thatwhich is received in step 340 (FIG. 4), but may not include the virtualvehicle skin itself. In such a case, the vehicle can then query one ormore remote servers, such as remote servers 82, for the virtual vehicleskin. After receiving the virtual vehicle skin, the vehicle may storethe virtual vehicle skin into memory, such as memory 38, for use at alater time, such as upon receiving another virtual vehicle skin request.

Additionally, after obtaining the virtual vehicle skin, the vehicle canprocess the skin or various components of the skin based on AR/VR devicelocation and position information. For example, the AR/VR device viewcan depend on a location and position of the AR/VR device and, thus, thevirtual vehicle skin can be modified in light of these attributes beforebeing sent to the AR/VR device. While the AR/VR device can take care ofmost AR/VR processing, the vehicle can assist the AR/VR device byproviding only those components of the virtual vehicle skin that arewithin view of a user at the AR/VR device's position. For example, thegraphic components 156 and 158 of the back-side of seats 120 and 130would most likely not be needed by a user sitting in seat 120 or 130.

In one embodiment, the position of the user can be obtained throughusing various VSMs, such as wireless communications device 30 and/or avehicle occupancy detector. The wireless communications device 30 canreceive information, such as information from a user, that specifies aseat within the vehicle that the AR/VR device user is located. Or, thevehicle occupancy detector that is coupled to a bed of a vehicle seatcan determine that only a single user is in the vehicle besides thedriver and, thus, the vehicle can determine the location of the singleuser and AR/VR device. Or, in other embodiments, a plurality of antennas(e.g., directional antennas) can be used throughout the vehicle toobtain angle of arrival and/or angle of departure information, as wellas signal strength information (e.g., relative received signal strength(RSSI)). The method 400 continues to step 440.

In step 440, the virtual vehicle skin is sent to the AR/VR device. Asthose skilled in the art will appreciate, the virtual vehicle skin canbe digitally represented as an array of bits that can be used torepresent the graphics, metadata, and other information of the virtualvehicle skin. In one embodiment, the virtual vehicle skin can be sentfrom the vehicle to the AR/VR device via an established SRWC connection,such as the one discussed above. The data representing the virtualvehicle skin may be larger than a packet size used in the SRWCconnection and, in such a case, the virtual vehicle skin can becommunicated to the AR/VR device using multiple packets. Upon receivingthe packets comprising the virtual vehicle skin, the AR/VR device maythen reconstruct the virtual vehicle skin.

In another embodiment, the vehicle can send virtual vehicle skin accessinformation to the AR/VR device. This virtual vehicle skin accessinformation can include information that enables the AR/VR device toobtain or derive the virtual vehicle skin. In one embodiment, thisinformation can include a vehicle identifier (e.g., VIN), a virtualvehicle skin token, and/or universal resource locator (URL) (oruniversal resource identifier (URI)) that can be used to download thevirtual vehicle skin from one or more remote servers, such as remoteservers 82. For example, the virtual vehicle skin token can be generatedat the time of purchasing or associating a particular virtual vehicleskin with a particular vehicle or user account (see step 330 of FIG. 4),or the virtual vehicle skin token can be generated upon receiving avirtual vehicle skin request at the remote server. The virtual vehicleskin token can be sent with a vehicle identifier (e.g., VIN) to a remoteserver, which can then verify the token and the vehicle identifier and,in response to the verification, the remote server can respond with thevirtual vehicle skin. In some embodiments, the token and vehicleidentifier can be included in the URL or URI sent to the AR/VR device.Once the virtual vehicle skin is obtained by the AR/VR device, themethod 400 ends.

With reference to FIG. 6, there is shown a method 500 of presentingvirtual vehicle skins over a corresponding area of a vehicle using anAR/VR device. Method 500 can be carried out by an AR/VR device, such assmartphone 14 and/or AR/VR goggles 16. The AR/VR device can include oneor more digital cameras that are configured to capture images or videoof an area surrounding the user, such as an area to the front of theuser. For example, when the AR/VR device is worn by the user, a cameraincluded on the AR/VR device may face an area in front of the user. TheAR/VR device can then capture images or video of this area and,additionally, when the vehicle or a component of the vehicle is withinthe field of view of the AR/VR device camera (i.e., the “AR/VR devicefield of view”), the AR/VR device can present the virtual vehicle skinover the vehicle or vehicle component. And, in other embodiments, theAR/VR device can receive captured image or video data from otherdevices, or may receive and/or present virtual background graphics sothat the user experiences a full virtual reality (VR) visual effect.

The method 500 can begin with step 510, wherein an indication that thevehicle supports virtual vehicle skins is received at the AR/VR device.In one embodiment, this indication can be a virtual vehicle skinavailability message, as discussed above (step 410 of FIG. 5). Asmentioned above, this can include one or more virtual vehicle skinidentifiers, a service ID, or other information that can be used toindicate that virtual vehicle skins are available for use with an AR/VRdevice, particularly for use in overlaying graphics onto an area of thevehicle within an AR view. After this message is received, the method500 continues to step 520.

In step 520, a virtual vehicle skin request is sent from the AR/VRdevice to the vehicle. This request was discussed above with respect tostep 420 of method 400 (FIG. 5). As mentioned above, the request caninclude various information, such as the virtual vehicle skin identifierthat identifies which virtual vehicle skin the AR/VR device desires toobtain. In other embodiments, a vehicle user may configure their vehicleto enable/disable certain virtual vehicle skins, which can includeenabling a single virtual vehicle skin at a given time. Thus, in such acase, the virtual vehicle skin request may not specify a particularvirtual vehicle skin, but may request whichever virtual vehicle skin ispresently enabled to be used with AR/VR devices for augmenting thevehicle or vehicle components. The method 500 then continues to step530.

In step 530, the virtual vehicle skin is received at the AR/VR device.The virtual vehicle skin may be embodied in various data files that canbe sent using one or more messages to the vehicle and, after the filesare received, the AR/VR device may then compile the files together toobtain the virtual vehicle skin. In one embodiment, the vehicle can savethe virtual vehicle skin to a memory device included in the AR/VRdevice, and may also send a virtual vehicle skin confirmation message tothe vehicle (or to the remote server) verifying that the virtual vehicleskin has been received.

As mentioned above, the virtual vehicle skin may be received from eitherthe vehicle or a remote server. In the latter case, the AR/VR device maypresent a token, which can be received by the AR/VR device from thevehicle, to the remote server and, in response thereto, the remoteserver can send the virtual vehicle skin to the AR/VR device via landnetwork 76 and/or cellular carrier system 70. In a particularembodiment, in response to receiving the virtual vehicle skin request,the vehicle can retrieve the requested (or determined) virtual vehicleskin from a remote server and, thereafter, the vehicle can forward thevirtual vehicle skin to the AR/VR device using an SRWC connection or awired connection. The method 500 then continues to step 540.

In step 540, the AR/VR device presents the virtual vehicle skin on thedisplay of the AR/VR device. In many embodiments, the virtual vehicleskin is presented over images or video captured by the AR/VR device andpresented in a region of the captured images or video in which thevehicle or vehicle components reside. The AR/VR device can use variousaugmented reality techniques to present the virtual vehicle skin on thedisplay in a proper manner. In one embodiment, the virtual vehicle skincan include numerous graphics that comprise the virtual vehicle skin.Each of these graphics can be associated with a particular component orarea of the vehicle.

In one embodiment, the virtual vehicle skin can include a plurality ofvirtual vehicle skin components, each of which corresponds to aparticular area or component of the vehicle. For example, with referenceto FIG. 3, there is shown various vehicle components, including seats120,130, A-pillars 140,142, and various trim components around centerconsole 106 and shifter 112. Virtual vehicle skin components 150-160 canbe presented over these components and, in some embodiments, AR/VRdevice can use image processing and/or object recognition techniques, aswell as other augmented reality (AR) techniques, to identify a vehiclecomponent, along with one or more attributes of the vehicle component orthe AR/VR device field of view. These virtual vehicle skin components150-160 can then be presented based on this information. For example,the AR/VR device can identify a position and size of the driver seat120, as well as viewing angle of the AR/VR device. Then, using thisinformation, the AR/VR device can then determined a position and size ofthe virtual vehicle component 156, which can be the same or at leastbased on the position and size of the seat 120. The AR/VR device canthen render these graphics 156 in this determined region. Moreover, theviewing angle of the AR/VR device can also be used as a basis fortransforming and/or distorting the virtual vehicle component 156 so thatit appears natural to the user.

In one embodiment, the AR/VR device can use an accelerometer, as well asradio communications with the vehicle, to determine a position,orientation, and/or viewing angle of the AR/VR device within thevehicle. Additionally, or alternatively, the AR/VR device can use imageprocessing techniques to recognize certain objects within the field ofview of the AR/VR device camera to corroborate the location and/ororientation of the AR/VR device within the vehicle. For example, thevehicle interior may contain certain attributes or components (i.e.,“virtual vehicle skin markers”) that are readily recognizable throughuse of image processing techniques. Information concerning these virtualvehicle skin markers can be included in the virtual vehicle skin dataand used to inform the AR/VR device of where the various virtual vehicleskin components are to be displayed. Based on the component informationincluded as part of the virtual vehicle skin, the AR/VR device canposition the various virtual vehicle skin components within the displayso that the AR components correspond to the components within thevehicle. For example, the AR/VR device can determine a position withinthe video output (which is to be displayed on the display of the AR/VRdevice) that a particular component should be located and, thereafter,the AR/VR device can then include the AR component at the determinedlocation within the video output. The video output may include real-lifevideo or images that are gathered by a camera on the AR/VR device, aswell as any virtual vehicle skin components that may be renderedthereon. Moreover, as the AR/VR device is repositioned or reoriented,the AR/VR device can update the video output to reflect an augmentedreality (AR) output that corresponds to the new field of view of theAR/VR device.

In other embodiments, the virtual vehicle skin can include virtualexterior environment graphics that can be used to add additional visualeffects to areas outside the vehicle and normally visible through awindow. For example, the vehicle can include virtual exteriorenvironment graphics as a part of the virtual vehicle skin and, then,the AR/VR device can: determine exterior areas within the field of viewof the AR/VR device; and, then, render the virtual exterior environmentgraphics within these regions so that the user of the AR/VR deviceexperiences a visual effect of being in a different or modifiedenvironment. For example, the virtual exterior environment graphics candepict outer space so that the vehicle appears to be traveling in outerspace. In one embodiment, the regions outside the vehicle can beentirely virtual graphics—that is, none of the captured images or videoof these exterior regions are retained for viewing on the display of theAR/VR device. The method 500 then ends.

In one embodiment, the method 300, the method 400, and/or the method 500(or parts of these various methods) can be implemented in a computerprogram (or “application”) embodied in a computer readable medium andincluding instructions usable by one or more processors of one or morecomputers of one or more systems. The computer program may include oneor more software programs comprised of program instructions in sourcecode, object code, executable code or other formats; one or morefirmware programs; or hardware description language (HDL) files; and anyprogram related data. The data may include data structures, look-uptables, or data in any other suitable format. The program instructionsmay include program modules, routines, programs, objects, components,and/or the like. The computer program can be executed on one computer oron multiple computers in communication with one another.

The program(s) can be embodied on computer readable media (such asmemory that is a part of the remote servers 82 (method 300), memory 38(method 400), memory on the AR/VR device (method 500)), which can benon-transitory and can include one or more storage devices, articles ofmanufacture, or the like. Exemplary computer readable media includecomputer system memory, e.g. RAM (random access memory), ROM (read onlymemory); semiconductor memory, e.g. EPROM (erasable, programmable ROM),EEPROM (electrically erasable, programmable ROM), flash memory; magneticor optical disks or tapes; and/or the like. The computer readable mediummay also include computer to computer connections, for example, whendata is transferred or provided over a network or another communicationsconnection (either wired, wireless, or a combination thereof). Anycombination(s) of the above examples is also included within the scopeof the computer-readable media. It is therefore to be understood thatthe method can be at least partially performed by any electronicarticles and/or devices capable of carrying out instructionscorresponding to one or more steps of the disclosed method.

It is to be understood that the foregoing is a description of one ormore embodiments of the invention. The invention is not limited to theparticular embodiment(s) disclosed herein, but rather is defined solelyby the claims below. Furthermore, the statements contained in theforegoing description relate to particular embodiments and are not to beconstrued as limitations on the scope of the invention or on thedefinition of terms used in the claims, except where a term or phrase isexpressly defined above. Various other embodiments and various changesand modifications to the disclosed embodiment(s) will become apparent tothose skilled in the art. All such other embodiments, changes, andmodifications are intended to come within the scope of the appendedclaims.

As used in this specification and claims, the terms “e.g.,” “forexample,” “for instance,” “such as,” and “like,” and the verbs“comprising,” “having,” “including,” and their other verb forms, whenused in conjunction with a listing of one or more components or otheritems, are each to be construed as open-ended, meaning that the listingis not to be considered as excluding other, additional components oritems. Other terms are to be construed using their broadest reasonablemeaning unless they are used in a context that requires a differentinterpretation. In addition, the term “and/or” is to be construed as aninclusive OR. Therefore, for example, the phrase “A, B, and/or C” is tobe interpreted as covering any one or more of the following: “A”; “B”;“C”; “A and B”; “A and C”; “B and C”; and “A, B, and C.”

1. A method of configuring an augmented reality/virtual reality (AR/VR)device to augment image or video data using a virtual vehicle skin,wherein the method is carried out by vehicle electronics included withina vehicle, the method comprising: establishing a connection to the AR/VRdevice using a wireless communications device included in the vehicleelectronics of the vehicle; and sending a virtual vehicle skin responseto the AR/VR device via the established connection, wherein the AR/VRdevice is configured to obtain image or video data and to display thevirtual vehicle skin over the obtained image or video data so that thevirtual vehicle skin overlays a portion of the vehicle, and wherein thevirtual vehicle skin response includes the virtual vehicle skin orvirtual vehicle skin access information that can be used by the AR/VRdevice to obtain the virtual vehicle skin.
 2. The method of claim 1,wherein the virtual vehicle skin is configured to be inserted intocaptured image or video data such that the virtual vehicle skin covers aportion of an interior cabin of the vehicle.
 3. The method of claim 1,further comprising the steps of: receiving a first virtual vehicle skinrequest from the AR/VR device; sending a second virtual vehicle skinrequest to one or more remote servers in response to receiving the firstvirtual vehicle skin request; and receiving the virtual vehicle skin orthe virtual vehicle skin access information.
 4. The method of claim 1,wherein the virtual vehicle skin response includes the virtual vehicleskin.
 5. The method of claim 4, wherein the digital representation ofthe virtual vehicle skin is obtained from memory included in the vehicleelectronics.
 6. The method of claim 1, wherein the virtual vehicle skinresponse includes virtual vehicle skin access information that includesinformation usable by the AR/VR device for obtaining the virtual vehicleskin from a remote server.
 7. The method of claim 6, wherein the virtualvehicle skin access information includes a virtual vehicle skin tokenthat, when presented to the remote server, results in the remote serversending the virtual vehicle skin to the AR/VR device.
 8. The method ofclaim 1, wherein the AR/VR device is a pair of wearable electronicaugmented reality/virtual reality (AR/VR) goggles configured forvisually presenting augmented reality or virtual reality media contentto a user.
 9. The method of claim 1, wherein the AR/VR device is apersonal AR/VR device.
 10. A method of configuring an augmentedreality/virtual reality (AR/VR) device to augment image or video datausing a virtual vehicle skin, the method comprising: capturing image orvideo data using a camera included on the AR/VR device; receiving avirtual vehicle skin advertisement message from a vehicle, wherein thevehicle uses a wireless communications device to send the virtualvehicle skin advertisement message; in response to receiving the virtualvehicle skin advertisement message, sending a virtual vehicle skinrequest, wherein the virtual vehicle skin request includes informationfrom the virtual vehicle skin advertisement message; and receiving avirtual vehicle skin response, wherein the virtual vehicle skin responseincludes a digital representation of the virtual vehicle skin orinformation that can be used by the AR/VR device to derive the digitalrepresentation of the virtual vehicle skin.
 11. The method of claim 10,wherein the virtual vehicle skin request is sent to the vehicle, andwherein the virtual vehicle skin response is sent by the vehicle. 12.The method of claim 11, wherein the virtual vehicle skin responseincludes virtual vehicle skin access information, and wherein the methodfurther comprises the steps of: sending a second virtual vehicle skinrequest to a remote server using the virtual vehicle skin accessinformation; and receiving a second virtual vehicle skin response thatincludes the digital representation of the virtual vehicle skin.
 13. Themethod of claim 10, wherein the virtual vehicle skin includes aplurality of virtual vehicle skin components, and wherein the methodfurther comprises the steps of: determining to display at least onevirtual vehicle skin component of the plurality of virtual vehicle skincomponents based on the image or video data; and rendering at least partof the at least one virtual vehicle skin component on a display of theAR/VR device.
 14. The method of claim 13, further comprising the stepof, before rendering the at least part of the at least one virtualvehicle skin component on the display of the AR/VR device, modifying theat least one virtual vehicle skin component based on the captured imageor video data.
 15. The method of claim 10, wherein the virtual vehicleskin includes virtual exterior environment graphics, and wherein themethod further comprises the steps of: determining exterior areas thatare exterior to the vehicle and within the captured image or video data;and rendering the virtual exterior environment graphics within theexterior areas.
 16. The method of claim 10, further comprising the stepof determining an orientation of the AR/VR device using an accelerometerincluded on the AR/VR device.
 17. The method of claim 10, wherein thevehicle is configured to determine a position of the AR/VR device withinthe vehicle and, thereafter, to obtain the virtual vehicle skin based onthe position of the AR/VR device.